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Department  of  Defense’s  Waiver  of  Competitive  Prototyping  Requirement  for  the  Air 
Force’s  B-2  Defensive  Management  System  Modernization  Program 

The  Weapon  Systems  Acquisition  Reform  Act  of  2009,  as  amended  (WSARA),  requires  the 
Secretary  of  Defense  to  modify  guidance  to  ensure  that  the  acquisition  strategy  for  each  major 
defense  acquisition  program  provides  for  competitive  prototypes  before  Milestone  B  approval — 
which  authorizes  entry  into  system  development — unless  the  Milestone  Decision  Authority 
waives  the  requirement.1  Competitive  prototyping,  which  involves  commercial,  government,  or 
academic  sources  producing  early  prototypes  of  weapon  systems  or  critical  subsystems,  can 
help  Department  of  Defense  (DOD)  programs  reduce  technical  risk,  refine  requirements, 
validate  designs  and  cost  estimates,  and  evaluate  manufacturing  processes  prior  to  making 
major  commitments  of  resources.  It  can  also  help  reduce  the  time  it  takes  to  field  a  system,  and 
as  a  result,  reduce  its  acquisition  cost.  WSARA  states  that  the  Milestone  Decision  Authority  may 
waive  the  competitive  prototyping  requirement  only  on  the  basis  that  (1 )  the  cost  of  producing 
competitive  prototypes  exceeds  the  expected  life-cycle  benefits  (in  constant  dollars)  of 
producing  such  prototypes,  including  the  benefits  of  improved  performance  and  increased 
technological  and  design  maturity  that  may  be  achieved  through  competitive  prototyping;  or  (2) 
but  for  such  a  waiver,  DOD  would  be  unable  to  meet  critical  national  security  objectives. 

WSARA  also  provides  that  whenever  a  Milestone  Decision  Authority  authorizes  a  waiver  of  the 
competitive  prototyping  requirement  on  the  basis  of  what  WSARA  describes  as  “excessive 
cost,”  the  Milestone  Decision  Authority  is  required  to  submit  notification  of  the  waiver,  together 
with  the  rationale,  to  the  Comptroller  General  of  the  United  States  at  the  same  time  it  is 
submitted  to  the  congressional  defense  committees.  WSARA  further  provides  that  no  later  than 
60  days  after  receipt  of  a  notification  of  a  waiver,  we  are  mandated  to  review  the  rationale  for 
the  waiver  and  submit  a  written  assessment  of  that  rationale  to  the  congressional  defense 
committees.2 


Vub.  L.  No.  1 11-23,  §  203(a),  as  amended  by  the  Ike  Skelton  National  Defense  Authorization  Act  for  Fiscal  Year 
2011,  Pub.  L.  No.  111-383,  §  813.  DOD  modified  its  guidance  related  to  the  operation  of  its  acquisition  system 
through  Directive-Type  Memorandum  (DTM)  09-027,  “Implementation  of  Weapon  Systems  Acquisition  Reform  Act  of 
2009,”  (Dec.  4,  2009,  incorporating  Change  4,  Jan.  11, 2013).  This  DTM  also  defined  major  defense  acquisition 
programs  as  those  estimated  by  DOD  to  require  an  eventual  total  expenditure  for  research,  development,  test,  and 
evaluation  or  for  procurement-including  all  planned  increments-of  more  than  $365  million  or  more  than  $2.19  billion, 
respectively,  in  fiscal  year  2000  constant  dollars.  DOD  issued  an  Interim  Instruction  5000.02,  “Operation  of  the 
Defense  Acquisition  System,”  on  November  25,  2013  which  cancelled  DTM  09-027  and  modified  the  thresholds  for 
major  defense  acquisition  programs.  However  for  purposes  of  this  report  we  used  DTM  09-027  because  the 
competitive  prototyping  waiver  for  the  B-2  Defensive  Management  System  Modernization  program  was  completed 
prior  to  issuance  of  the  Interim  DOD  Instruction.  The  Milestone  Decision  Authority  for  major  defense  acquisition 
programs  is  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics;  the  head  of  a  DOD 
component;  or,  if  delegated,  the  component  acquisition  executive. 

2Pub.  L.  No.  111-23,  §  203(b)(1)  &  (2). 
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On  December  3,  2013,  we  received  notice  from  DOD  that  it  had  waived  the  competitive 
prototyping  requirement  for  the  Air  Force’s  B-2  Defensive  Management  System  Modernization 
(DMS-M)  program.  The  Air  Force’s  B-2  bomber  is  a  low-observable,  long-range  strike  aircraft 
capable  of  entering  heavily  defended  areas  to  deliver  both  conventional  and  nuclear  weapons. 
The  defensive  management  system  detects,  identifies,  and  locates  enemy  radar  systems  and 
provides  real-time  threat  avoidance,  threat  warning,  and  threat  situational  awareness 
information  to  the  aircrew.  The  DMS-M  program  is  expected  to  upgrade  the  aircraft's  analog 
defensive  management  system  to  a  digital  capability,  update  pilot  displays,  enhance  the  in-flight 
capability  to  avoid  unanticipated  air  defense  threats,  and  improve  frequency  coverage  and  the 
reliability  and  maintainability  of  the  system.  The  DMS-M  consists  of  three  critical  subsystems — 
antenna,  electronic  support  measures,  and  avionics  and  graphics  processors — that  will  be 
integrated  onto  existing  B-2  aircraft.  The  program  is  in  the  second  of  two  technology 
development  phases  and  plans  to  gain  approval  to  enter  system  development  in  late  fiscal  year 
2015. 

In  this  report,  we  assess  DOD’s  rationale  for  waiving  the  competitive  prototyping  requirement  for 
the  B-2  DMS-M  program  and  the  analysis  used  to  support  it.  To  conduct  our  assessment,  we 
compared  the  rationale  in  the  waiver  to  the  WSARA  requirement  to  determine  the  extent  to 
which  the  waiver  is  consistent  with  the  statute.  In  addition,  we  reviewed  the  Air  Force’s  cost- 
benefit  analysis,  which  provides  the  data  and  assumptions  on  which  the  waiver  is  based,  the 
technology  development  strategy,  and  other  relevant  documentation.  We  did  not  independently 
verify  the  Air  Force’s  data  on  cost  and  benefits.  We  also  submitted  written  questions  to  B-2 
program  officials  to  clarify  information  in  program  documentation,  as  necessary. 

We  conducted  this  performance  audit  from  March  2014  to  April  2014  in  accordance  with 
generally  accepted  government  auditing  standards.  Those  standards  require  that  we  plan  and 
perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives. 


Results  in  Brief 

DOD’s  rationale  for  waiving  WSARA’s  competitive  prototyping  requirement  for  the  B-2  DMS-M 
program  addresses  one  of  the  two  bases  provided  in  the  statute;  namely  that  the  cost  of 
producing  competitive  prototypes  exceeds  the  expected  life-cycle  benefits  (in  constant  dollars) 
of  producing  the  prototypes.  The  Air  Force’s  B-2  DMS-M  program  is  well  into  technology 
development  and  according  to  the  cost-benefit  analysis  supporting  the  waiver,  there  is  little 
opportunity  to  gain  significant  cost  savings  by  prototyping.  According  to  the  Air  Force,  the  prime 
contractor  has  already  held  competitions  and  selected  suppliers  that  will  leverage  existing, 
technically  mature  subsystems  for  two  of  three  DMS-M  critical  subsystems.  The  prime 
contractor  has  also  conducted  competitive  prototyping  for  antennas,  which  are  not  yet 
technically  mature,  to  reduce  technical,  cost,  and  schedule  risk.  In  its  cost-benefit  analysis,  the 
Air  Force  concluded  that  it  will  achieve  significant  operations  and  support  cost  reductions 
through  its  strategy  of  leveraging  existing  subsystems  for  the  DMS-M  and  any  additional 
improvements  from  prototyping  would  be  on  the  margins.  In  the  waiver,  DOD  concluded  that  the 
Air  Force’s  prototyping  cost-benefit  analysis  was  reasonable.  We  also  found  that  the  Air  Force’s 
cost-benefit  analysis  was  consistent  with  key  principles  in  DOD’s  policy  on  economic  analysis, 
which  states  that  each  feasible  alternative  for  meeting  an  objective  must  be  considered  and  its 
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life-cycle  costs  and  benefits  evaluated.3  The  Air  Force  analysis  examined  four  different 
scenarios  including  the  system-  and  subsystem-level  prototyping  options  outlined  in  WSARA 
and  DOD’s  implementing  guidance.  The  Air  Force  concluded  these  strategies  would  increase 
the  program’s  development  costs  by  between  $28.2  million  and  $524.8  million  (in  base  year 
201 1  dollars)  depending  on  the  type  and  number  of  prototypes.  The  Air  Force  also  estimated 
that  the  various  prototyping  strategies  could  achieve  an  estimated  $1 .3  million  to  $6.3  million  (in 
base  year  201 1  dollars)  in  life-cycle  benefits  by  improving  the  reliability  of  the  system  or  key 
subsystems. 


Waiver  Rationale  Is  Consistent  with  WSARA  and  the  Supporting  Analysis  Considered  a 
Reasonable  Set  of  Prototyping  Alternatives 

DOD’s  rationale  for  waiving  WSARA’s  competitive  prototyping  requirement  for  the  B-2  DMS-M 
program  addressed  one  of  the  two  bases  provided  for  a  waiver  in  the  statute,  namely  that  the 
cost  of  producing  competitive  prototypes  exceeds  the  expected  life-cycle  benefits,  including  the 
benefits  of  improved  performance  and  increased  technological  and  design  maturity  that  may  be 
achieved  through  competitive  prototyping.  The  Air  Force’s  B-2  DMS-M  program  is  well  into 
technology  development  and  according  to  the  cost  benefit  analysis  supporting  the  waiver,  there 
is  little  opportunity  to  gain  significant  cost  savings  by  prototyping.  During  technology 
development,  the  prime  contractor  Northrop  Grumman  held  competitions  and  selected  suppliers 
for  critical  DMS-M  subsystems.  According  to  the  Air  Force,  the  chosen  suppliers  for  the 
electronic  support  measures  and  avionics  and  graphics  processors  subsystems  will  leverage 
existing,  technically  mature  subsystems  and  will  also  produce  single  prototypes  of  these 
subsystems  during  technology  development.  During  technology  development,  the  prime 
contractor  also  conducted  competitive  prototyping  for  B-2  DMS-M  antennas,  which  are  not  yet 
technically  mature,  to  reduce  technical,  cost,  and  schedule  risk.  According  to  the  Air  Force, 
there  were  four  competitive  prototypes  produced  for  one  part  of  the  antenna  subsystem  and 
three  competitive  prototypes  produced  for  the  other  part.  In  its  cost-benefit  analysis,  the  Air 
Force  concluded  that  its  strategy  of  leveraging  existing  subsystems  for  the  DMS-M  will  achieve 
a  47  percent  or  $155.8  million  (in  base  year  201 1  dollars)  reduction  in  operations  and  support 
costs  over  the  current  system  and  any  additional  improvements  from  prototyping  would  be  on 
the  margins.  We  also  noted  that  the  Air  Force  plans  to  use  a  cost-type  contract  for  system 
development,  which,  according  to  DOD  acquisition  regulations,  requires  a  written  determination 
by  the  Milestone  Decision  Authority  that  a  program  is  so  complex  and  technically  challenging 
that  it  would  not  be  practicable  to  reduce  program  risk  to  a  level  that  would  permit  the  use  of  a 
fixed-price  type  contract.4  The  Air  Force  recommended  a  cost-type  contract  because  of 
uncertainty  related  to  software  development,  system  integration,  testing,  and  potential  funding 
volatility. 

In  the  waiver,  DOD  found  reasonable  the  Air  Force’s  cost-benefit  analysis,  which  examined 
multiple  prototyping  strategies  including  system- and  subsystem-level  prototyping.  The  Air 
Force’s  cost-benefit  approach  was  consistent  with  key  principles  in  DOD’s  policy  on  economic 
analysis,  which  states  that  each  feasible  alternative  for  meeting  an  objective  must  be 


3DOD  Instruction  7041.3,  Economic  Analysis  for  Decisionmaking,  Enel.  3,  para.  E3.1.1.  (Nov.  7,  1995). 

4Defense  Federal  Acquisition  Regulation  Supplement  §  234.004(2)(i)(C)(1) .  Cost-reimbursement  contracts,  also 
known  as  cost-type  contracts,  provide  for  payment  of  allowable  incurred  costs,  to  the  extent  prescribed  in  the 
contract.  This  contract  type  places  most  of  the  risk  on  the  government,  which  may  pay  more  than  budgeted  should 
incurred  costs  be  more  than  expected  when  the  contract  is  signed. 
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considered  and  its  life-cycle  costs  and  benefits  evaluated.5  In  all,  the  Air  Force  examined  four 
different  scenarios,  including  the  system-  and  subsystem-level  prototyping  options  outlined  in 
WSARA  and  DOD’s  implementing  guidance,  and  compared  the  life-cycle  cost  of  each 
prototyping  scenario  to  the  potential  life-cycle  benefits.  The  Air  Force  concluded  these 
strategies  would  increase  the  program’s  development  costs  by  between  $28.2  million  to  $524.8 
million  (in  base  year  201 1  dollars)  depending  on  the  type  and  number  of  prototypes.  The  Air 
Force’s  prototyping  cost  estimates  were  based  primarily  on  the  actual  and  planned  cost  of  the 
program’s  two  technology  development  phases.  For  example,  the  Air  Force  assumed  a  second 
contractor  would  need  to  develop  a  DMS-M  system  prototype  at  the  same  cost  as  Northrop 
Grumman’s  system  delivered  during  technology  development.  In  addition,  the  system-level 
competitive  prototyping  scenario  included  $54  million  for  data  rights  because  access  to 
proprietary  Northrop  Grumman  data  is  needed  to  act  as  the  system  integrator  for  B-2  related 
efforts.  According  to  the  Air  Force,  this  estimate  was  developed  using  historic  cost  estimating 
factors.  In  practice,  Northrop  Grumman  is  unwilling  to  provide  a  price  to  the  government  for  the 
data,  so  this  prototyping  scenario  might  not  be  feasible.  The  Air  Force  also  estimated  that  the 
various  prototyping  strategies  could  achieve  an  estimated  $1.3  million  to  $6.3  million  (in  base 
year  201 1  dollars)  in  life-cycle  benefits  by  improving  the  reliability  of  the  system  or  key 
subsystems.  According  to  the  Air  Force,  it  also  considered  the  positive  impacts  of  prototyping 
with  respect  to  integration  and  production  costs,  but  it  did  not  include  a  dollar  value  for  these 
benefits  in  any  of  its  prototyping  scenarios. 


5DOD  Instruction  7041.3,  Economic  Analysis  for  Decisionmaking,  Enel.  3,  para.  E3.1.1.  (Nov.  7,  1995). 
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Agency  Comments  and  Our  Evaluation 

We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  response,  DOD  stated  that  it  had  no 
comments. 


We  are  sending  copies  of  this  report  to  interested  congressional  committees,  the  Secretary  of 
Defense  and  the  Secretary  of  the  Air  Force.  In  addition,  the  report  will  be  available  at  no  charge 
on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions,  please  contact  me  at  (202)  51 2-4841  or 
sullivanm@gao.gov.  Contact  points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs 
may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to  this 
report  were  Ron  Schwenn,  Assistant  Director;  Kristine  Hassinger;  Matthew  B.  Lea;  Kenneth  E. 
Patton;  Carol  D.  Petersen;  and  Don  M.  Springman. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the 
United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety 
without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be 
necessary  if  you  wish  to  reproduce  this  material  separately. 
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